Learning Non-Linear Functions for Text Classification

In this paper, we show that generative classifiers are capable of learning non-linear decision boundaries and that non-linear generative models can outperform a number of linear classifiers on some text categorization tasks. We first prove that 3-layer multinomial hierarchical generative (Bayesian) classifiers, under a particular independence assumption, can only learn the same linear decision boundaries as a multinomial naive Bayes classifier. We then go on to show that making a different independence assumption results in nonlinearization, thereby enabling us to learn non-linear decision boundaries. We finally evaluate the performance of these non-linear classifiers on a series of text classification tasks.